Fix container-related misconfigurations in release build tasks #201

per1234 · 2024-11-12T03:35:14Z

Please check if the PR fulfills these requirements

The PR has no duplicates (please search among the Pull Requests
before creating one)
The PR follows our contributing guidelines
[N/A] Tests for the changes have been added (for bug fixes / features)

What kind of change does this PR introduce?

Bug fix.

What is the current behavior?

DistTasks.yml contains the tasks used to produce the release builds of the project for each of the host targets. The builds are produced in Docker containers.

Some regressions were introduced in these task at the time the project's Go version was bumped to 1.21.5 (which includes bumping the versions of the images used by the tasks: 1ccd378 / #174).

Non-existent container tag referenced in the Linux_ARMv6 task.
Repository configuration incompatible for use in a container with modern Git versions due to "dubious ownership" (go vcs stamping will break with new git elastic/golang-crossbuild#232).

These regressions would cause the failure of the "Release" workflow run when it is triggered by a new release of the project.

What is the new behavior?

Use Debian 11 in `Linux_ARMv6` release build task

The Linux_ARMv6 task must use a specific version of Debian in the container:

arduino-language-server/DistTasks.yml

Lines 147 to 174 in 5444eb5

    
           # We are experiencing the following error with ARMv6 build: 
        
           # 
        
           #   # github.com/arduino/arduino-cli 
        
           #   net(.text): unexpected relocation type 296 (R_ARM_V4BX) 
        
           #   panic: runtime error: invalid memory address or nil pointer dereference 
        
           #   [signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x51ae53] 
        
           # 
        
           #   goroutine 1 [running]: 
        
           #   cmd/link/internal/loader.(*Loader).SymName(0xc000095c00, 0x0, 0xc0000958d8, 0x5a0ac) 
        
           #           /usr/local/go/src/cmd/link/internal/loader/loader.go:684 +0x53 
        
           #   cmd/link/internal/ld.dynrelocsym2(0xc000095880, 0x5a0ac) 
        
           #           /usr/local/go/src/cmd/link/internal/ld/data.go:777 +0x295 
        
           #   cmd/link/internal/ld.(*dodataState).dynreloc2(0xc007df9800, 0xc000095880) 
        
           #           /usr/local/go/src/cmd/link/internal/ld/data.go:794 +0x89 
        
           #   cmd/link/internal/ld.(*Link).dodata2(0xc000095880, 0xc007d00000, 0x60518, 0x60518) 
        
           #           /usr/local/go/src/cmd/link/internal/ld/data.go:1434 +0x4d4 
        
           #   cmd/link/internal/ld.Main(0x8729a0, 0x4, 0x8, 0x1, 0xd, 0xe, 0x0, 0x0, 0x6d7737, 0x12, ...) 
        
           #           /usr/local/go/src/cmd/link/internal/ld/main.go:302 +0x123a 
        
           #   main.main() 
        
           #           /usr/local/go/src/cmd/link/main.go:68 +0x1dc 
        
           #   Error: failed building for linux/armv6: exit status 2 
        
           # 
        
           # This seems to be a problem in the go builder 1.16.x that removed support for the R_ARM_V4BX instruction: 
        
           #    https://github.com/golang/go/pull/44998 
        
           #    https://groups.google.com/g/golang-codereviews/c/yzN80xxwu2E 
        
           # 
        
           # Until there is a fix released we must use a recent gcc for Linux_ARMv6 build, so for this 
        
           # build we select the debian10 based container.

The Debian version is defined via the image tag. Previously, Debian 10 was used, and a tag of the Go 1.18.3 image was available for this Debian version. However, the maintainers of the image did not produce a Debian 10 variant of the Go 1.21.5 image, so the use of that tag caused the task to fail:

https://github.com/per1234/arduino-language-server/actions/runs/11772916519/job/32789011179#step:4:36

Unable to find image 'docker.elastic.co/beats-dev/golang-crossbuild:1.21.5-armel-debian10' locally
docker: Error response from daemon: manifest for docker.elastic.co/beats-dev/golang-crossbuild:1.21.5-armel-debian10 not found: manifest unknown: manifest unknown.
See 'docker run --help'.
task: Failed to run task "dist:Linux_ARMv6": exit status 125

A Debian 11 variant of the image is available, and this version of Debian is also suitable for release builds. So the solution is to update the image tag referenced in the task to the Debian 11 tag.

Configure repository for compatibility with modern Git versions in release build containers

As a security measure (see CVE-2022-24765), starting from 2.30.3 Git requires the repository folder to be owned by the operating system user's account. Due to it having been checked out outside the container, the repository does not meet this requirement inside the container. An older version of Git was installed in the Go 1.18.3 Docker image, so this was not a problem before the bump, but a newer version is used in the Go 1.21.5 containers, which causes some tasks to fail (elastic/golang-crossbuild#232):

https://github.com/per1234/arduino-language-server/actions/runs/11772976565/job/32789161736#step:4:148

error obtaining VCS status: exit status 128
	Use -buildvcs=false to disable VCS stamping.
Error: failed building for linux/armv6: exit status 1
failed building for linux/armv6: exit status 1
task: Failed to run task "dist:Linux_ARMv6": exit status 1

https://github.com/per1234/arduino-language-server/actions/runs/11773011125/job/32789254088#step:4:146

error obtaining VCS status: exit status 128
	Use -buildvcs=false to disable VCS stamping.
Error: failed building for darwin/amd64: exit status 1
failed building for darwin/amd64: exit status 1
task: Failed to run task "dist:macOS_64bit": exit status 1

https://github.com/per1234/arduino-language-server/actions/runs/11773011125/job/32789254192#step:4:161

error obtaining VCS status: exit status 128
	Use -buildvcs=false to disable VCS stamping.
Error: failed building for darwin/arm64: exit status 1
failed building for darwin/arm64: exit status 1
task: Failed to run task "dist:macOS_ARM64": exit status 1

The solution is to configure Git to allow the use of the repository, despite the "dubious ownership" of its folder. This is done via the safe.directory Git configuration variable.

This approach is already in use in Arduino CLI's release build tasks: arduino/arduino-cli@5a5ae94 / arduino/arduino-cli#2103

Other information

In order to facilitate the review of this pull request, I performed a demonstration release with the proposed changes in my fork:

https://github.com/per1234/arduino-language-server/actions/runs/11789922878

https://github.com/per1234/arduino-language-server/releases/tag/0.0.0-rc.12

`DistTasks.yml` contains the tasks used to produce the release builds of the project for each of the host targets. The builds are produced in Docker containers. A regression was introduced in the `Linux_ARMv6` task at the time the project's Go version was bumped to 1.21.5. This task must use a specific version of Debian in the container, which is defined via the image tag. Previously, Debian 10 was used, and a tag of the Go 1.18.3 image was available for this Debian version. However, the maintainers of the image did not produce a Debian 10 variant of the Go 1.21.5 image, so the use of that tag caused the task to fail: ``` Unable to find image 'docker.elastic.co/beats-dev/golang-crossbuild:1.21.5-armel-debian10' locally docker: Error response from daemon: manifest for docker.elastic.co/beats-dev/golang-crossbuild:1.21.5-armel-debian10 not found: manifest unknown: manifest unknown. See 'docker run --help'. task: Failed to run task "dist:Linux_ARMv6": exit status 125 ``` A Debian 11 variant of the image is available, and this version of Debian is also suitable for release builds. So the solution is to update the image tag referenced in the task to the Debian 11 tag.

…lease build containers `DistTasks.yml` contains the tasks used to produce the release builds of the project for each of the host targets. The builds are produced in Docker containers. A regression was introduced in several of the tasks at the time the project's Go version was bumped to 1.21.5. As a security measure (see CVE-2022-24765), starting from 2.30.3 Git requires the repository folder to be owned by the operating system user's account. Due to it having been checked out outside the container, the repository does not meet this requirement inside the container. An older version of Git was installed in the Go 1.18.3 Docker image, so this was not a problem before the bump, but a newer version is used in the Go 1.21.5 containers, which causes some tasks to fail: ``` error obtaining VCS status: exit status 128 Use -buildvcs=false to disable VCS stamping. Error: failed building for linux/armv6: exit status 1 failed building for linux/armv6: exit status 1 task: Failed to run task "dist:Linux_ARMv6": exit status 1 ``` The solution is to configure Git to allow the use of the repository, despite the "dubious ownership" of its folder. This is done via the `safe.directory` Git configuration variable.

alessio-perugini

super 🚀

per1234 added 2 commits November 11, 2024 19:00

per1234 added topic: infrastructure Related to project infrastructure type: imperfection Perceived defect in any part of project labels Nov 12, 2024

per1234 requested review from cmaglie and alessio-perugini November 12, 2024 03:35

per1234 self-assigned this Nov 12, 2024

alessio-perugini approved these changes Nov 12, 2024

View reviewed changes

per1234 merged commit 1a68dfb into arduino:main Nov 13, 2024
8 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix container-related misconfigurations in release build tasks #201

Fix container-related misconfigurations in release build tasks #201

per1234 commented Nov 12, 2024 •

edited

Loading

alessio-perugini left a comment

	# We are experiencing the following error with ARMv6 build:
	#
	# # github.com/arduino/arduino-cli
	# net(.text): unexpected relocation type 296 (R_ARM_V4BX)
	# panic: runtime error: invalid memory address or nil pointer dereference
	# [signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x51ae53]
	#
	# goroutine 1 [running]:
	# cmd/link/internal/loader.(*Loader).SymName(0xc000095c00, 0x0, 0xc0000958d8, 0x5a0ac)
	# /usr/local/go/src/cmd/link/internal/loader/loader.go:684 +0x53
	# cmd/link/internal/ld.dynrelocsym2(0xc000095880, 0x5a0ac)
	# /usr/local/go/src/cmd/link/internal/ld/data.go:777 +0x295
	# cmd/link/internal/ld.(*dodataState).dynreloc2(0xc007df9800, 0xc000095880)
	# /usr/local/go/src/cmd/link/internal/ld/data.go:794 +0x89
	# cmd/link/internal/ld.(*Link).dodata2(0xc000095880, 0xc007d00000, 0x60518, 0x60518)
	# /usr/local/go/src/cmd/link/internal/ld/data.go:1434 +0x4d4
	# cmd/link/internal/ld.Main(0x8729a0, 0x4, 0x8, 0x1, 0xd, 0xe, 0x0, 0x0, 0x6d7737, 0x12, ...)
	# /usr/local/go/src/cmd/link/internal/ld/main.go:302 +0x123a
	# main.main()
	# /usr/local/go/src/cmd/link/main.go:68 +0x1dc
	# Error: failed building for linux/armv6: exit status 2
	#
	# This seems to be a problem in the go builder 1.16.x that removed support for the R_ARM_V4BX instruction:
	# https://github.com/golang/go/pull/44998
	# https://groups.google.com/g/golang-codereviews/c/yzN80xxwu2E
	#
	# Until there is a fix released we must use a recent gcc for Linux_ARMv6 build, so for this
	# build we select the debian10 based container.

Fix container-related misconfigurations in release build tasks #201

Fix container-related misconfigurations in release build tasks #201

Conversation

per1234 commented Nov 12, 2024 • edited Loading

Please check if the PR fulfills these requirements

What is the current behavior?

What is the new behavior?

Use Debian 11 in Linux_ARMv6 release build task

Configure repository for compatibility with modern Git versions in release build containers

Other information

alessio-perugini left a comment

Choose a reason for hiding this comment

per1234 commented Nov 12, 2024 •

edited

Loading

Use Debian 11 in `Linux_ARMv6` release build task